Heavy inertial particles in turbulent flows gain energy slowly but lose it rapidly

We present an extensive numerical study of the time irreversibility of the dynamics of heavy inertial particles in three-dimensional, statistically homogeneous and isotropic turbulent flows. We show that the probability density function (PDF) of the increment, $W(\tau)$, of a particle's energy over a time-scale $\tau$ is non-Gaussian, and skewed towards negative values. This implies that, on average, particles gain energy over a period of time that is longer than the duration over which they lose energy. We call this $\textit{slow gain}$ and $\textit{fast loss}$. We find that the third moment of $W(\tau)$ scales as $\tau^3$, for small values of $\tau$. We show that the PDF of power-input $p$ is negatively skewed too; we use this skewness ${\rm Ir}$ as a measure of the time-irreversibility and we demonstrate that it increases sharply with the Stokes number ${\rm St}$, for small ${\rm St}$; this increase slows down at ${\rm St} \simeq 1$. Furthermore, we obtain the PDFs of $t^+$ and $t^-$, the times over which $p$ has, respectively, positive or negative signs, i.e., the particle gains or loses energy. We obtain from these PDFs a direct and natural quantification of the the slow-gain and fast-loss of the particles, because these PDFs possess exponential tails, whence we infer the characteristic loss and gain times $t_{\rm loss}$ and $t_{\rm gain}$, respectively; and we obtain $t_{\rm loss} < t_{\rm gain}$, for all the cases we have considered. Finally, we show that the slow-gain in energy of the particles is equally likely in vortical or strain-dominated regions of the flow; in contrast, the fast-loss of energy occurs with greater probability in the latter than in the former

How long do particles spend in vortical regions in turbulent flows?

We obtain the probability distribution functions (PDFs) of the time that a Lagrangian tracer or a heavy inertial particle spends in vortical or strain-dominated regions of a turbulent flow, by carrying out direct numerical simulation (DNS) of such particles advected by statistically steady, homogeneous and isotropic turbulence in the forced, three-dimensional, incompressible Navier-Stokes equation. We use the two invariants, $Q$ and $R$, of the velocity-gradient tensor to distinguish between vortical and strain-dominated regions of the flow and partition the $Q-R$ plane into four different regions depending on the topology of the flow; out of these four regions two correspond to vorticity-dominated regions of the flow and two correspond to strain-dominated ones. We obtain $Q$ and $R$ along the trajectories of tracers and heavy inertial particles and find out the time $\mathrm{t_{pers}}$ for which they remain in one of the four regions of the $Q-R$ plane. We find that the PDFs of $\mathrm{t_{pers}}$ display exponentially decaying tails for all four regions for tracers and heavy inertial particles. From these PDFs we extract characteristic times scales, which help us to quantify the time that such particles spend in vortical or strain-dominated regions of the flow

Hierarchical Average Reward Policy Gradient Algorithms (Student Abstract)

Option-critic learning is a general-purpose reinforcement learning (RL) framework that aims to address the issue of long term credit assignment by leveraging temporal abstractions. However, when dealing with extended timescales, discounting future rewards can lead to incorrect credit assignments. In this work, we address this issue by extending the hierarchical option-critic policy gradient theorem for the average reward criterion. Our proposed framework aims to maximize the long-term reward obtained in the steady-state of the Markov chain defined by the agent's policy. Furthermore, we use an ordinary differential equation based approach for our convergence analysis and prove that the parameters of the intra-option policies, termination functions, and value functions, converge to their corresponding optimal values, with probability one. Finally, we illustrate the competitive advantage of learning options, in the average reward setting, on a grid-world environment with sparse rewards